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„ . " J"/? MAILING DA TE of this communication appears on th cover sheet with the correspondence address » — 
Period for Reply r 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- ^^^J^J^^ available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely'filed 
after SIX (6) MONTHS from the mailing date of this communication. 

" I LnS f°7 eP '. y SpeCifie r "J 10 !! 8 iS '!u SS * han thirtV (30) dayS ' 3 reply Within the statutor V minimum ofthirt V W davs will be considered timely. 

' l^m^^^nVJ^I^^ ma ,T Um f atU !^ Peri ° d WI " apP ' y and Wi " GXpire SIX (6) MONTHS from the maili "9 da < e o^his communication. 

- Failure o reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 USC5 1 33) 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). ' * ,culK ' 0d "Y 

Status 

1)D Responsive to communication(s) filed on . 



2a)D This action is FINAL. 2b)E| This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O G 213 
Disposition of Claims 

4) E3 Claim(s) 1-55 is/are pending in the application. 

4a) Of the above claim(s) 46-55 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) IEI Claim(s) ±45 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) |3 Claim(s) 46-55 are subject to restriction and/or election requirement. 
Application Papers 

9) D The specification is objected to by the Examiner. 
10)13 The drawing(s) filed on 27 November 2000 is/are: a)D accepted or b)E3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 !)□ The proposed drawing correction filed on is: a)Q approved b)Q disapproved by the Examiner 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)0 All b)D Some*c)Q None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 119(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 

Attachment(s) 

1) |3 Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-1 52) 

3) [23 Information Disclosure Statement(s) (PTO-1 449) Paper No(s) 2 . 6) □ Other: 
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DETAILED ACTION 

Election/Restrictions 

1 . Restriction to one of the following inventions is required under 35U.S.C. 121: 

I. Claims 1-45, drawn to an audiovisual presentation that replaces selected reference 
portions with replacement objects, classified in class 345, subclass 716. 

II. Claims 46-55, drawn to a video processing system, classified in class 345, 
subclass 419-475. 

The inventions are distinct, each from the other because of the following reasons: 

2. Inventions I and II are related as subcombinations disclosed as usable together in a single 
combination. The subcombinations are distinct from each other if they are shown to be 
separately usable. In the instant case, invention I has separate utility such as producing an 
audiovisual presentation that substitutes images without requiring the particular video processing 
in invention II. The video processing of invention I could be carried out by other means than 
those described in invention II. See MPEP § 806.05(d). 

3. Because these inventions are distinct for the reasons given above and the search required 
for Group I is not required for Group II, restriction for examination purposes as indicated is 
proper. 

4. Because these inventions are distinct for the reasons given above and have acquired a 
separate status in the art because of their recognized divergent subject matter, restriction for 
examination purposes as indicated is proper. 
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5. During a telephone conversation with Attorney David Sitrick on 5/22/03 a provisional 
election was made with traverse to prosecute the invention of Group I, claims 1-45. Affirmation 
of this election must be made by applicant in replying to this Office action. Claims 46-55 are 
withdrawn from further consideration by the examiner, 37 CFR 1 .142(b), as being drawn to a 
non-elected invention. 

Drawings 

6. The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they 
do not include the following reference sign(s) mentioned in the description: the reference 
numerals in Fig. 12 are in the 1 100s, but starting with the paragraph on line 6 of page 40, the 
items in Fig. 12 are described with reference numerals in the 1200s. A proposed drawing 
correction or corrected drawings are required in reply to the Office action to avoid abandonment 
of the application. The objection to the drawings will not be held in abeyance. 

Claim Rejections - 35 USC §102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1(2) of such treaty in the English language. 

8. Claims 1- are rejected under 35 U.S.C. 102(e) as being clearly anticipated by U.S. Patent 
No. 6,504,546 to Cosatto et al. (hereinafter Cosatto). 
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Referring to claims 1, 26, and 37, Cosatto discloses a system and method for producing 
an integrated display audiovisual presentation with a source of first audiovisual presentation. 
See col. 3, lines 34-53, which describe how video is taken of a model along with an audio track. 
The first audiovisual presentation comprises at least one reference point and at least one 
reference object, wherein at least a selected one reference point is associated with a location on a 
selected one reference object. See col. 5, lines 25-40, which describe how facial parts are 
associated with their appropriate location on the face. 

Cosatto discloses a source of user audiovisual information, wherein the user audiovisual 
information comprises at least one replacement point and at least one replacement object image, 
wherein at least a selected one replacement point is associated with a location on a selected 
replacement object image. See col. 5, lines 41-64, which describe how bitmap images from 
recorded video (user audiovisual information) are used replace parts of the face (replacement 
parts) at an appropriate location on a face. 

Cosatto discloses correlation means responsive to the first audiovisual presentation and 
the user audiovisual information, for correlating at least one selected reference point (location of 
facial features) with at least one selected replacement point (facial features) and association 
means responsive to the correlation means, for associating a selected one reference object (i.e. 
mouth) with a selected one replacement object image (i.e. new mouth bitmap). See col. 3, lines 
34-53, col. 4, lines 30-43, and Figs. 1-7. 

Cosatto discloses compositing means responsive to the first audiovisual presentation, the 
user audiovisual information, the association means, and the correlation means, for compositing 
at least one replacement object image with the first audiovisual presentation, producing an 
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integrated display audiovisual presentation, wherein at least a selected portion of the replacement 
object image replaces a selected portion of the first audiovisual presentation. See Fig. 7, which 
shows facial features integrated into a separate base face, replacing the previous facial features. 
Also, see col. 6, lines 7-20. 

Referring to claims 9-10, the first audiovisual presentation of Cosatto comprises a time- 
ordered sequence of video images digitized and encoded into an encoded digital signal, that 
comprises a plurality of Motion Picture Expert's Group motion vectors, wherein the correlation 
means is further responsive to at least a selected one of the plurality of MPEG motion vectors. 
See col. 53-56, which discusses how the facial parts are handled by a model for the MPEG4 
subsystem. 

Referring to claim 2, the first audiovisual presentation of Cosatto further comprises a 
video input signal. See col. 3, line 38 and col. 4, line 15. 

Referring to claim 3, the video input signal of Cosatto is a time-ordered sequence of 
video images. See col. 6, line 66 - col. 7, line 3. Also, the animations of the video signal are 
time-ordered to match the audio track. 

Referring to claims 4, 1 1, 27, and 38, the correlation means of Cosatto further comprises 
recognition means responsive to the video input signal, for recognizing at least a selected one 
reference object within at least a selected one video image (the first audiovisual presentation; 
encoded digital signal) of the time-ordered sequence. Col. 7, line 14 - col. 8, line 15 describes 
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the recognition system of Cosatto, including how facial features (selected reference objects) are 
located within the video. 

Referring to claims 5, 12, and 28, the recognition means of Cosatto further recognizes at 
least one reference point on the associated reference object. See col. 7, lines 33-39. 

Referring to claims 6, 29, and 39, the recognition means of Cosatto comprises image 
recognition means. See col. 7, line 14 - col. 8, line 15. 

Referring to claims 7-8, the first audiovisual presentation of Cosatto further comprises 
reference object information that identifies and is associated with a selected one reference object. 
See col. 7, lines 40-61, which describe how reference object information (i.e. color and texture 
analysis, size and position - reference point) is associated with and identifies facial features 
(selected reference objects). 

Referring to claims 13, 30, and 40, Cosatto discloses the correlation means further 
comprises first-order object transformation means for transforming at least a selected one 
recognized reference point responsive to the selected one of the plurality of MPEG motion 
vectors. See col. 13, lines 60-65 which discuss morphing (transforming) and col. 14, lines 53-66, 
which discusses how the facial parts are handled by a model for the MPEG4 subsystem. 

Referring to claims 14-15, 3 1, and 41, the compositing means of Cosatto further 
comprises image transform means (i.e. morphing; col. 13, lines 60-65) for transforming the 
selected one replacement object image (facial feature) associated with the selected one reference 
object. 
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Referring to claims 16-18 and 32, the user audiovisual information of Cosatto further 
comprises user object geometric information. See col. 1 1, lines 5-7, which describe how the 
samples of facial features contain geometric information. 

The correlating means of Cosatto further comprises recognition means for identifying a 
selected one reference object within the first audiovisual presentation (col. 7, lines 40-61) and the 
compositing means further comprises geometric transformation means for geometrically 
transforming the user object geometric information responsive to the recognition means. See 
col. 10, lines 37-53 and col. 13, lines 22-34. 

The compositing means of Cosatto further comprises geometric transformation means for 
geometrically transforming the user object geometric information producing transformed user 
object geometric information, responsive to the selected one reference object. See col. 6, lines 7- 
20, where Cosatto acknowledges the ability the change the shape and orientation of objects. 

Referring to claims 19-20, 33, and 43, the replacement object image of Cosatto comprises 
texture map image data representative of at least one selected user object as viewed from at least 
one predefined viewing position relative to the selected user object and the compositing means 
selects a portion of the texture map image data responsive to the correlation means. See col. 6, 
lines 7-12. Also, see col. 1, lines 62-64. 

Referring to claims 21-22 and 34, the compositing means of Cosatto further comprises 
geometric transformation means for geometrically transforming the user object geometric 
information producing transformed user object geometric information, responsive to the selected 
one reference object. See col. 6, lines 7-20, where Cosatto acknowledges the ability the change 
the shape and orientation of objects. 
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The geometric transformation means of Cosatto further comprises mapping means for 
mapping the selected portion of the texture map image data onto the transformed user object 
geometric information producing a replacement object image. See col. 6, lines 7-20. 

Referring to claims 23, 35, and 45, the user audiovisual information of Cosatto further 
comprises a time-ordered sequence of user images and the association means further selectively 
associates a plurality of the time-ordered sequence of user images each respectively with selected 
ones of the time-ordered sequence of video images (prior to the replacing). See col. 3, lines 44- 
60, which describe how the user images are selected to coincide with an audio track in a time- 
ordered sequence to provide realistic lip and head movements. 

Referring to claims 24 and 36, the audiovisual presentation of Cosatto further comprises a 
time-ordered sequence of video masks, and the correlation means further comprises selection 
means for selecting a portion of the first audiovisual presentation responsive to at least a selected 
one of the video masks. See col. 15, lines 1-9. 

Referring to claim 25, the selection means of Cosatto is further responsive to at least a 
selected one of the MPEG motion vectors. See col. 14, line 55. 

Referring to claim 42, Cosatto discloses the replacement object further comprises user 
object geometric information that is transformed, producing transformed user object information, 
responsive to the recognition of at least one selected reference object within the first audiovisual 
presentation, prior to the replacing. See col. 6, lines 7-20, where Cosatto acknowledges the 
ability the change the shape and orientation of objects. Also, see col. 6, lines 38-45. 

Referring to claim 44, the selected portion of the texture map of Cosatto is mapped onto 
transformed user object geometric information, prior to the replacing. See col 6, lines 7-20. 
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Conclusion 



9. The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 CF.R. § 1 . 1 1 1 (c) to consider 
these references fully when responding to this action. The documents cited therein teach image 
substitution, such as changing clothing on models of users, changing faces on predefined 
characters or dolls and integrating real video data with virtual data. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Shawn M. Becker whose telephone number is'703-305-7756. 
The examiner can normally be reached on M-T 8:00 - 5:30 and alternating Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John W. Cabeca can be reached on 703-305-31 16. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-746-7239 for regular 
communications and 703-745-7238 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-305-3900. 
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TECHNOLOGY CENTER 21 0* 



